AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.04705	1.55
	2_DecisionTree	Decision Tree	logloss	0.378191	32.28
the best	3_Default_Xgboost	Xgboost	logloss	0.222508	67.93
	4_Default_NeuralNetwork	Neural Network	logloss	0.359378	8.73
	5_Default_RandomForest	Random Forest	logloss	0.344678	38.16
	Ensemble	Ensemble	logloss	0.222508	0.61

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

7.7 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.727363	0.92234	0.979263	0	0	0.403226	0.876356	0.505365	0.848691	0.359378
recall	0.865089	0.960456	0.992991	0	0	0.0764526	0.876356	0.482498	0.876356	0.359378
f1-score	0.79027	0.941012	0.986079	0	0	0.128535	0.876356	0.474316	0.85291	0.359378
support	845	2807	428	5	12	327	0.876356	4424	4424	0.359378

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	731	89	2	0	0	23
Labeled as Emergency	95	2696	2	0	0	14
Labeled as Newborn	0	3	425	0	0	0
Labeled as Not Available	0	5	0	0	0	0
Labeled as Trauma	2	10	0	0	0	0
Labeled as Urgent	177	120	5	0	0	25

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

36.9 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.728601	0.899137	0.993007	0	0	0.695652	0.870253	0.552733	0.85715	0.344678
recall	0.826036	0.965444	0.995327	0	0	0.0489297	0.870253	0.472623	0.870253	0.344678
f1-score	0.774265	0.931111	0.994166	0	0	0.0914286	0.870253	0.465162	0.84161	0.344678
support	845	2807	428	5	12	327	0.870253	4424	4424	0.344678

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	698	145	0	0	0	2
Labeled as Emergency	92	2710	1	0	0	4
Labeled as Newborn	0	1	426	0	0	1
Labeled as Not Available	0	5	0	0	0	0
Labeled as Trauma	1	11	0	0	0	0
Labeled as Urgent	167	142	2	0	0	16

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Elective (Fold 1)

Dependence Emergency (Fold 1)

Dependence Newborn (Fold 1)

Dependence Not Available (Fold 1)

Dependence Trauma (Fold 1)

Dependence Urgent (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

31.0 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.595668	0.974874	0.990698	0	0	0.727273	0.856465	0.548085	0.881928	0.378191
recall	0.976331	0.898468	0.995327	0	0	0.0489297	0.856465	0.486509	0.856465	0.378191
f1-score	0.73991	0.935113	0.993007	0	0	0.0916905	0.856465	0.459953	0.837495	0.378191
support	845	2807	428	5	12	327	0.856465	4424	4424	0.378191

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	825	18	0	0	0	2
Labeled as Emergency	280	2522	2	0	0	3
Labeled as Newborn	0	1	426	0	0	1
Labeled as Not Available	0	5	0	0	0	0
Labeled as Trauma	3	9	0	0	0	0
Labeled as Urgent	277	32	2	0	0	16

Learning curves

Decision Tree

Tree #1

Rules

if (Emergency Department Indicator > 0.5) and (APR MDC Description <= 22.5) and (Zip Code - 3 digits <= 27.5) then class: Emergency (proba: 98.89%) | based on 5,501 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight <= 250.0) and (APR Medical Surgical Description <= 0.5) then class: Elective (proba: 41.6%) | based on 2,238 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight <= 250.0) and (APR Medical Surgical Description > 0.5) then class: Elective (proba: 76.89%) | based on 2,012 samples

if (Emergency Department Indicator > 0.5) and (APR MDC Description <= 22.5) and (Zip Code - 3 digits > 27.5) then class: Emergency (proba: 96.56%) | based on 1,746 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight > 250.0) and (CCS Diagnosis Code <= 220.0) then class: Newborn (proba: 99.07%) | based on 1,296 samples

if (Emergency Department Indicator > 0.5) and (APR MDC Description > 22.5) and (Facility Id <= 1716.0) then class: Emergency (proba: 85.19%) | based on 432 samples

if (Emergency Department Indicator > 0.5) and (APR MDC Description > 22.5) and (Facility Id > 1716.0) then class: Urgent (proba: 84.85%) | based on 33 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight > 250.0) and (CCS Diagnosis Code > 220.0) then class: Urgent (proba: 53.85%) | based on 13 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Elective (Fold 1)

Dependence Emergency (Fold 1)

Dependence Newborn (Fold 1)

Dependence Not Available (Fold 1)

Dependence Trauma (Fold 1)

Dependence Urgent (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

0.8 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0	0.634494	0	0	0	0	0.634494	0.105749	0.402582	1.04705
recall	0	1	0	0	0	0	0.634494	0.166667	0.634494	1.04705
f1-score	0	0.776379	0	0	0	0	0.634494	0.129397	0.492608	1.04705
support	845	2807	428	5	12	327	0.634494	4424	4424	1.04705

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	0	845	0	0	0	0
Labeled as Emergency	0	2807	0	0	0	0
Labeled as Newborn	0	428	0	0	0	0
Labeled as Not Available	0	5	0	0	0	0
Labeled as Trauma	0	12	0	0	0	0
Labeled as Urgent	0	327	0	0	0	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	1

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.826797	0.961089	0.993023	1	1	0.768595	0.925859	0.924918	0.92445	0.222508
recall	0.898225	0.967937	0.997664	0.8	0.25	0.568807	0.925859	0.747106	0.925859	0.222508
f1-score	0.861032	0.964501	0.995338	0.888889	0.4	0.653779	0.925859	0.793923	0.923138	0.222508
support	845	2807	428	5	12	327	0.925859	4424	4424	0.222508

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	759	52	0	0	0	34
Labeled as Emergency	66	2717	2	0	0	22
Labeled as Newborn	1	0	427	0	0	0
Labeled as Not Available	0	1	0	4	0	0
Labeled as Trauma	0	9	0	0	3	0
Labeled as Urgent	92	48	1	0	0	186

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: multi:softprob
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: mlogloss
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

66.6 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.826797	0.961089	0.993023	1	1	0.768595	0.925859	0.924918	0.92445	0.222508
recall	0.898225	0.967937	0.997664	0.8	0.25	0.568807	0.925859	0.747106	0.925859	0.222508
f1-score	0.861032	0.964501	0.995338	0.888889	0.4	0.653779	0.925859	0.793923	0.923138	0.222508
support	845	2807	428	5	12	327	0.925859	4424	4424	0.222508

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	759	52	0	0	0	34
Labeled as Emergency	66	2717	2	0	0	22
Labeled as Newborn	1	0	427	0	0	0
Labeled as Not Available	0	1	0	4	0	0
Labeled as Trauma	0	9	0	0	3	0
Labeled as Urgent	92	48	1	0	0	186

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Elective (Fold 1)

Dependence Emergency (Fold 1)

Dependence Newborn (Fold 1)

Dependence Not Available (Fold 1)

Dependence Trauma (Fold 1)

Dependence Urgent (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back